Repository navigation
fix: redact every column that names a user, verify unrecorded MySQL columns - #52
Merged
gjtorikian merged 5 commits intoNov 6, 2026
Merged
Conversation
…olumns `redact --user` in Postgres, MySQL and MongoDB erased rows by `user_id` only. An invitation names a user as its inviter and as the user who accepted it, so those rows, and the archived and quarantined events carrying those payloads, kept the subject's id and email after the command reported success. Every column and payload field that names a user is matched now, as in the ClickHouse package. MySQL recorded a column with no `_columns` record as current without a look at the table, so after a schema applied by hand a changed path was recorded as installed with the column still reading the old path. The catalog's clause now goes onto a temporary table, and `SHOW CREATE TABLE` renders that and the live table with the same server, so the two compare as text: a match is recorded, anything else is drift.
[High impact] The PR appears safe to merge; the previous event-deletion issue is fixed. SummaryExpands user deletion across all four destinations and checks MySQL columns that have no saved definition.
Reviews (4) · Last reviewed commit: "test(engine): expect the catalog's user ..." · Reviewed by Greptile |
…tion Which columns hold a user id is a fact about the catalog, not about any database, yet each destination carried its own list. A column can now declare what it references (`user_id` references `users`), and the engine answers `userReferenceColumns(stream)` for the mirror rows and `userReferenceFields()` for the archived payloads; the four `redact` implementations ask it instead of repeating the list. The MySQL README's "no DDL after the first init" promise now says what checking an unrecorded column does: a temporary table created and dropped within the session, which is session state, not a schema change.
…om the streams An extra column named `user_id` counted as naming a user before and still does, without declaring anything. The payload paths `redact` scrubs archived and quarantined events by come from the effective streams, extra columns included, as path steps rather than top-level field names, so a nested reference such as `metadata.owner_user_id` is scrubbed too. Each destination builds its JSON path from the steps: a parameterised array for Postgres, the JSON path helpers elsewhere.
…treams are selected The archive keeps every event type the tail subscribed to, so a run mirroring only `users` still archives `invitation.accepted` with the subject's id in it. The payload paths `redact` scrubs by now come from the whole catalog, with the selection adding only its extra columns.
gjtorikian
deleted the
fix/redact-user-references-and-unrecorded-columns
branch
November 6, 2026 15:29
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
redact --userin Postgres, MySQL and MongoDB erased rows byuser_idonly. An invitation names a user as its inviter and as the user who accepted it, so those rows, and the archived and quarantined events carrying those payloads, kept the subject's id and email after the command reported success. Every column and payload field that names a user is matched now (user_id,inviter_user_id,accepted_user_id), the same fix the ClickHouse package got in feat(clickhouse): add the ClickHouse destination package #46._columnsrecord as current without a look at the table, so after a schema applied by hand frominit --print-sql, a changed path was recorded as installed with the column still reading the old path. The catalog's clause now goes onto a temporary table andSHOW CREATE TABLErenders that and the live table with the same server, so the two compare as text; a match is recorded and anything else is drift that--repairrebuilds. One pinned connection, since the temporary table lives in its session, and within theCREATE TEMPORARY TABLESgrant the README already lists. Postgres already did this through a temp table andpg_get_expr.Testing
Each package's redaction test now creates an invitation the subject accepted, one the subject sent and one unrelated, and expects two invitation rows erased, one left, and no archived event naming the subject. MySQL's drift suite covers the unrecorded column: the same catalog checks out and is recorded, a changed path is reported as drift and withholds the fingerprint, and
--repairrebuilds it. The three suites pass locally (Postgres on PGlite, MySQL 8.4, MongoDB memory server).